Papers by Emile Van Krieken
Mixtures of In-Context Learners (2025.acl-long)
Copied to clipboard
| Challenge: | In-context learning is sensitive to the choice of in-con context demonstrations and processing many demonstrations can be computationally demanding. |
| Approach: | They propose a method that uses subsets of demonstrations to train experts via ICL and learns a weighting function to merge their output distributions via gradient-based optimisation. |
| Outcome: | The proposed approach improves on 5 out of 7 classification datasets compared to strong baselines and reduces the inference time needed to achieve the same performance with fewer demonstrations. |
Self-Training Large Language Models for Tool-Use Without Demonstrations (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent work augmented LLMs with tools to mitigate factual inaccuracies and computational errors. |
| Approach: | They propose a method to synthesise tool-use traces using the LLM itself. |
| Outcome: | The proposed method improves performance on a long-tail knowledge task, but not on other datasets. |
Are We Done with MMLU? (2025.naacl-long)
Copied to clipboard
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, Pasquale Minervini
| Challenge: | MMLU is widely adopted but its ground truth errors obscure the true capabilities of LLMs. |
| Approach: | They propose a framework for identifying dataset errors using a novel error annotation protocol and a subset of 5,700 manually re-annotated questions. |
| Outcome: | The proposed framework is based on 5,700 re-annotated questions from the MMLU benchmark. |